Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Arbitrary Matrix Shapes #4

Closed
wants to merge 3 commits into from

Conversation

Xeratec
Copy link
Collaborator

@Xeratec Xeratec commented Sep 24, 2024

This PR introduces initial support for handling arbitrary matrix shapes by adjusting both the Python model and the Softmax hardware module.

Changes

Python Model:

  • Modified to golden model for arbitrary shapes by padding the inputs with zeros as needed.
  • Updated to selectively ignore padded input values during the softmax operation

Current Limitations

  • Non-zero biases are not yet supported (setting bias to zero for padded values is missing)
  • The HWPE version does not yet work correctly. There is most likely a bug
  • Only works with ReLU activation (controller for activation is not yet adjusted)
  • Feedforward and MatMul only work with one tile (controller is not yet adjusted)

ToDo

  • Set bias to zero for padded values
  • Fix HWPE tests with padded values
  • Adjust controller to handle feedforward and MatMul stages
  • Adjust GeLU activation to handle padded values

Important Changes:
- Change scaling of Softmax from 2**7-1 to 2**8-1

Current Limitations:
- Only works without biases
- Only works with ReLU activation
- FeedForward and MatMul do only work with one Tile
Changes:
- Add register for shape parameters

Current Limitations:
- Only works without biases
- Only works with ReLU activation
- FeedForward and MatMul do only work with one Tile
@Xeratec Xeratec self-assigned this Sep 24, 2024
@marcelkant marcelkant mentioned this pull request Oct 7, 2024
4 tasks
@Xeratec
Copy link
Collaborator Author

Xeratec commented Oct 7, 2024

@marcelkant Will continue the development on #5

@Xeratec Xeratec closed this Oct 7, 2024
@Xeratec Xeratec deleted the dev/padded_softmax_rebase branch November 4, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant